Predicting and Preventing Coordination Problems in Cooperative Q-learning Systems
نویسندگان
چکیده
We present a conceptual framework for creating Qlearning-based algorithms that converge to optimal equilibria in cooperative multiagent settings. This framework includes a set of conditions that are sufficient to guarantee optimal system performance. We demonstrate the efficacy of the framework by using it to analyze several well-known multi-agent learning algorithms and conclude by employing it as a design tool to construct a simple, novel multiagent learning algorithm.
منابع مشابه
Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems
In the framework of fully cooperative multi-agent systems, independent (non-communicative) agents that learn by reinforcement must overcome several difficulties to manage to coordinate. This paper identifies several challenges responsible for the non-coordination of independent agents: Pareto-selection, nonstationarity, stochasticity, alter-exploration and shadowed equilibria. A selection of mu...
متن کاملMultiagent Coordination in Cooperative Q-learning Systems
Many reinforcement learning architectures fail to learn optimal group behaviors in the multiagent domain. Although these coordination difficulties are often attributed to the non-Markovian environment created by the gradually-changing policies of concurrently learning agents, a careful analysis of the situation reveals an underlying problem structure which can cause suboptimal group policies ev...
متن کاملVoltage Coordination of FACTS Devices in Power Systems Using RL-Based Multi-Agent Systems
This paper describes how multi-agent system technology can be used as the underpinning platform for voltage control in power systems. In this study, some FACTS (flexible AC transmission systems) devices are properly designed to coordinate their decisions and actions in order to provide a coordinated secondary voltage control mechanism based on multi-agent theory. Each device here is modeled as ...
متن کاملBaselines for Joint-Action Reinforcement Learning of Coordination in Cooperative Multi-agent Systems
We report on an investigation of reinforcement learning techniques for the learning of coordination in cooperative multiagent systems. Specifically, we focus on a novel action selection strategy for Q-learning (Watkins 1989). The new technique is applicable to scenarios where mutual observation of actions is not possible. To date, reinforcement learning approaches for such independent agents di...
متن کاملImproving on the reinforcement learning of coordination in cooperative multi-agent systems
We report on an investigation of reinforcement learning techniques for the learning of coordination in cooperative multiagent systems. These techniques are variants of Q-learning (Watkins, 1989) that are applicable to scenarios where mutual observation of actions is not possible. To date, reinforcement learning approaches for such independent agents did not guarantee convergence to the optimal ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2007